Differentiating the Method of Conjugate Gradients

Conjugate gradient neural network in prediction of clay behavior and parameters sensitivities

The use of artificial neural networks has increased in many areas of engineering. In particular, this method has been applied to many geotechnical engineering problems and demonstrated some degree of success. A review of the literature reveals that it has been used successfully in modeling soil behavior, site characterization, earth retaining structures, settlement of structures, slope stabilit...

Lanczos and Linear Systems Lanczos and Linear Systems

abstract Lanczos's major contributions to the numerical solution of linear equations are contained in two papers: \An Iteration Method for the Solution of the Eigenvalue Problem of Linear Diierential and Integral Operators" and \Solutions of Linear Equations by Minimized Iterations ," the second of which contains the method of conjugate gradients. In this note we retrace Lanczos's journey from ...

Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models

The natural gradient method has been used effectively in conjugate Gaussian process models, but the non-conjugate case has been largely unexplored. We examine how natural gradients can be used in non-conjugate stochastic settings, together with hyperparameter learning. We conclude that the natural gradient can significantly improve performance in terms of wall-clock time. For illconditioned pos...

Parallel Preconditioned Conjugate-Gradients Methods on Transputer Networks

We show how highly efficient parallel implementations of basic linear algebra routines may be used as building blocks to implement efficient higher level algorithms. We discuss the solution of systems of linear equations using a preconditioned Conjugate-Gradients iterative method on a network of transputers. Results are presented for the solution of both dense and sparse systems; the latter bei...

The Method of Conjugate Gradients in Finite Element Applications